fix: add warning message for job-prefixed pipeline steps when no job name is provided #4371

svia3 · 2024-01-11T22:16:31Z

Issue #, if available:

Add warning message for when customers are using custom-prefixing in their pipelines and do NOT specify a job-name to prefix.
E.G. Cx fails to provide model-name here

tuning_step_prefix = prefix + "/Training_Artifacts"
model = Model(
    image_uri=image_uri,
    model_data=step_tuning.get_top_model_s3_uri(
        top_k=0, s3_bucket=bucket, prefix=tuning_step_prefix
    ),
    sagemaker_session=pipeline_session,
    role=role,
    name="A_Name" <<<<<<<<<<<<<<<<<<<<<<<<<<<<
)

results in this error

TypeError: expected string or byte like object

Description of changes:

Testing done:
-unit

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

rohangujarathi · 2024-01-11T22:30:35Z

src/sagemaker/workflow/utilities.py

@@ -41,6 +41,12 @@
    "if desired."
 )

+JOB_KEY_NONE_WARN_MSG_TEMPLATE = (
+    "The provided job field [{}] has not been specified. Please refer to the AWS Docs to "


can we add link to the docs as well?

sure! good call.

qidewenwhen · 2024-01-11T22:52:30Z

src/sagemaker/workflow/utilities.py

+    "The provided job field [{}] has not been specified. Please refer to the AWS Docs to "
+    "identify which field should be set to enable the custom-prefixing feature for jobs "
+    "created via a pipeline execution. "
+    "https://docs.aws.amazon.com/sagemaker/latest/dg/build-and-manage-access.html"


Is this the correct doc?
It's pointing to "IAM Access Management"

Yea its under the subsection. Customize access management for SageMaker Pipelines jobs

qidewenwhen · 2024-01-11T22:57:28Z

tests/unit/sagemaker/workflow/test_model_step.py

+        )
+        pipeline.definition()
+
+    assert "The provided job field [ModelName] has not been specified" in str(error.value)


Model step won't create a job so the "job field" here may be confusing.

How about: "Invalid input: the use_custom_job_prefix is set but the field [ModelName] has not been specified. Please ..."

We should highlight in what case the missing ModelName/job name would be invalid

Sure, I like this idea.

qidewenwhen · 2024-01-11T22:59:27Z

src/sagemaker/workflow/utilities.py

@@ -458,6 +465,8 @@ def trim_request_dict(request_dict, job_key, config):
        request_dict.pop(job_key, None)  # safely return null in case of KeyError
    else:
        if job_key in request_dict:
+            if request_dict[job_key] is None:


what if request_dict[job_key] is an empty string? is that a valid case?

Shouldnt be. Its dependent on all the class constructors resolving that field as a default to None. Should we add guardrails for that as well ? That would have to be manually inputted by the CX. aka model-name=""

mufaddal-rohawala · 2024-01-11T23:48:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9167a6a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-11T23:50:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 9167a6a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-11T23:51:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 9167a6a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T00:29:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 2ffab7a
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T00:37:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e3e7837
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T01:28:18Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e3e7837
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T01:31:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e3e7837
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T01:41:37Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e3e7837
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T03:19:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e3e7837
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T20:52:20Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: c3a0d04
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T21:43:15Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 7c854b9
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T22:33:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 7c854b9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T22:35:11Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 7c854b9
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-12T23:14:04Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e783a46
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-13T00:05:53Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e783a46
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-13T01:09:06Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e783a46
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-16T21:01:31Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: 5d1daa5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-16T21:02:50Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 5d1daa5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-16T21:06:23Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: 5d1daa5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-16T21:24:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 5d1daa5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-16T21:58:27Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 5d1daa5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2024-01-16T21:59:25Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (668e65d) 86.91% compared to head (5d1daa5) 86.79%.
Report is 1 commits behind head on master.

❗ Current head 5d1daa5 differs from pull request most recent head d855a8e. Consider uploading reports for the commit d855a8e to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4371      +/-   ##
==========================================
- Coverage   86.91%   86.79%   -0.13%     
==========================================
  Files        1197      380     -817     
  Lines      106707    35170   -71537     
==========================================
- Hits        92749    30526   -62223     
+ Misses      13958     4644    -9314

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mufaddal-rohawala · 2024-01-17T01:10:26Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: d855a8e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-17T02:00:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: d855a8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-17T02:03:17Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: d855a8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-17T02:14:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: d855a8e
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-17T02:27:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: d855a8e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

svia3 · 2024-01-17T19:57:49Z

/bot run unit-tests

…name is provided

mufaddal-rohawala · 2024-01-19T02:21:12Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-pr
Commit ID: e63702e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-19T02:21:49Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: e63702e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-19T02:23:46Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-notebook-tests
Commit ID: e63702e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-19T03:18:40Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: e63702e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mufaddal-rohawala · 2024-01-19T03:19:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: e63702e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com>

…ngestion. (#4413) * change: update image_uri_configs 12-13-2023 12:23:06 PST * change: update image_uri_configs 12-13-2023 14:04:54 PST * prepare release v2.200.1 * update development version to v2.200.2.dev0 * fix: Move func and args serialization of function step to step level (#4312) * fix: Add write permission to job output dirs for remote and step decorator running on non-root job user (#4325) * feat: Added update for model package (#4309) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * documentation: fix ModelBuilder sample notebook links (#4319) * feat: Use specific images for SMP v2 jobs (#4333) * Add check for smp lib * update syntax * Remove unused images * Update repo name and regions * Update account number * Update framework name and check for None distribution * Add unit tests for smp v2 uri * Check enabled * Remove logging * Add cuda version in uri * Update cu121 * Update syntax * Fix black check * Fix black --------- Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> * Fix: Updated js mb compression logic - ModelBuilder (#4294) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * documentation: SMP v2 doc updates (#1423) (#4336) * doc update for estimator distribution art * add note to the SMP doc and minor fixes * remove subnodes * rm all v1 content as documenting everything in aws docs * fix build errors * fix white spaces * rm smdistributed from TF estimator distribution * rm white spaces * add notes to TF estimator distribution * fix links * incorporate feedback * update example values * fix version numbers in the notes Co-authored-by: Miyoung <cmiyoung@amazon.com> * prepare release v2.201.0 * update development version to v2.201.1.dev0 * Fix: Add additional model builder telemetry (#4334) * move telemetry code to public * add additional test --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * feature: support remote debug for sagemaker training job (#4315) * feature: support remote debug for sagemaker training job * change: Replace update_remote_config with 2 helper methods for enable and disable respectively * change: add new argument enable_remote_debug to skip set of test_jumpstart_estimator_kwargs_match_parent_class * chore: add jumpstart support for remote debug --------- Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> * Update tblib constraint (#4317) * Fix: Fix job_objective type (#4303) * change: update image_uri_configs 12-21-2023 08:32:41 PST * prepare release v2.202.0 * update development version to v2.202.1.dev0 * Using logging instead of prints (#4133) * documentation: update issue template. (#4337) * change: update model path in local mode (#4296) * Update model path in local mode * Add test * change: update image_uri_configs 12-22-2023 06:17:35 PST * prepare release v2.202.1 * update development version to v2.202.2.dev0 * change: create role if needed in `get_execution_role` (#4323) * Create role if needed in get_execution_role * Add tests * Change: More pythonic tags (#4327) * Change: More pythonic tags * Fix broken tags * More tags formatting and add a test * Fix tests * Raise Exception for debug (#4344) Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * Change: Allow extra_args to be passed to uploader (#4338) * Change: Allow extra_args to be passed to uploader * Fix tests * Black * Fix test * Change: Drop py2 tag from the wheel as we don't support Python 2 (#4343) * Disable failed test in IR (#4345) * Disable failed test in IR * Fix format --------- Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * change: update image_uri_configs 12-25-2023 06:17:33 PST * feat: Supporting tbac in load_run (#4039) * feature: support local mode in SageMaker Studio (#1300) (#4347) * feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * prepare release v2.203.0 * update development version to v2.203.1.dev0 * change: update image_uri_configs 12-29-2023 06:17:34 PST * query hf api for model md (#4346) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * fix: skip failing integs (#4348) Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * change: TGI 1.3.3 (#4335) * prepare release v2.203.1 * update development version to v2.203.2.dev0 * feat: parallelize notebook search utils, add new operators (#4342) * feat: parallelize notebook search utils * chore: raise exception in notebook utils if thread has error * chore: improve variable name * fix: not passing region to get jumpstart bucket * chore: add sagemaker session to notebook utils * chore: address PR comments * feat: add support for includes, begins with, ends with * fix: pylint * feat: private util for model eula key * fix: unit tests, use verify_model_region_and_return_specs in notebook utils * Revert "feat: private util for model eula key" This reverts commit e2daefc. * chore: add search keywords to header * fix: change ConditionNot incorrect property Expression to Condition (#4351) * fix: Huggingface glue failing tests (#4367) * fix: Huggingface glue failing tests * fix: Sphinx doc build failure * fix: Huggingface glue failing tests * fix: failing sphinx tests * fix: failing sphinx tests * fix: failing black check * fix: sphinx doc errors * fix: sphinx doc errors * sphinx * black-format * sphinx * sphinx * sphinx --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> * fix: Add PyTorch 2.1.0 SM Training DLC to UNSUPPORTED_DLC_IMAGE_FOR_SM_PARALLELISM list (#4356) * add 2.1 unsupported smddp * formatting * feat: Support custom repack model settings (#4328) * change: update sphinx version (#4377) * change: update sphinx version * Update sphinx * change: Updates for DJL 0.26.0 release (#4366) * change: TGI NeuronX (#4375) * TGI NeuronX * Update * Update * fix: add warning message for job-prefixed pipeline steps when no job name is provided (#4371) Co-authored-by: svia3 <svia@amazon.com> * change: JumpStart - TLV region launch (#4379) * feat: add throughput management support for feature group (#4359) * feat: add throughput management support for feature group * documentation: add doc for feature group throughput config --------- Co-authored-by: Nilesh PS <psnilesh@amazon.com> * change: Enable galactus integ tests (#4376) * feat: Enable galactus integ tests * fix flake8 * fix doc8 * trying to see if it works with slow tests * small fixes in import error * fix missing import * try to remove some dependencies from requirement to see if pr test can be fixed * fix flake8 * Enable more tests * Add rerun annotation and further remove dependencies * comment out 2 integ tests * Remove local mode test for now * fix flake8 * prepare release v2.204.0 * update development version to v2.204.1.dev0 * fix: Add validation for empty ParameterString value in start local pipeline (#4354) * feat: Support selective pipeline execution for function step (#4372) * change: update image_uri_configs 01-24-2024 06:17:33 PST * fix: update get_execution_role_arn from metadata file if present (#4388) * fix: Support using PipelineDefinitionConfig in local mode (#4352) * fix: remove fastapi and uvicorn dependencies (#4365) They are not used in the codebase. Closes #4361 #4295 * prepare release v2.205.0 * update development version to v2.205.1.dev0 * change: TGI NeuronX 0.0.17 (#4390) * fix: Support PipelineVariable for ModelQualityCheckConfig attributes (#4353) * feat: Logic to detect hardware GPU count and aggregate GPU memory size in MiB (#4389) * Add logic to detect hardware GPU count and aggregate GPU memory size in MiB * Fix all formatting * Addressed PR review comments * Addressed PR Review messages * Addressed PR Review Messages * Addressed PR Review comments * Addressed PR Review Comments * Add integration tests * Add config * Fix integration tests * Include Instance Types GPU infor Config files * Addressed PR review comments * Fix unit tests * Fix unit test: 'Mock' object is not subscriptable --------- Co-authored-by: Jonathan Makunga <makung@amazon.com> * fix: fixed create monitoring schedule failing after validation error (#4385) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * Add collection type support for Feaure Group Ingestion. Add TargetStores support for PutRecord and Ingestion. * Remove merge conflicts. * Update the feature definition type * Black formatting * Fix Flake8 formatting * Fix Pylint * Fix Formatting. --------- Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com> Co-authored-by: ci <ci> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Keshav Chandak <keshav.chandak1995@gmail.com> Co-authored-by: Keshav Chandak <chakesh@amazon.com> Co-authored-by: stacicho <stacicho@amazon.com> Co-authored-by: Teng-xu <67929972+Teng-xu@users.noreply.github.com> Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> Co-authored-by: Gary Wang <38331932+gwang111@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> Co-authored-by: akrishna1995 <38850354+akrishna1995@users.noreply.github.com> Co-authored-by: Miyoung <cmiyoung@amazon.com> Co-authored-by: Xinyu Xie <xiexinyucrab@126.com> Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> Co-authored-by: martinRenou <martin.renou@gmail.com> Co-authored-by: Duc Trung Le <leductrungxf@gmail.com> Co-authored-by: ruiliann666 <141953824+ruiliann666@users.noreply.github.com> Co-authored-by: Ruilian Gao <ruiliann@amazon.com> Co-authored-by: ananth102 <ananthbashyam1@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: amzn-choeric <105388439+amzn-choeric@users.noreply.github.com> Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> Co-authored-by: Sirut Buasai <73297481+sirutBuasai@users.noreply.github.com> Co-authored-by: Sindhu Somasundaram <56774226+sindhuvahinis@users.noreply.github.com> Co-authored-by: Stephen Via <51342648+svia3@users.noreply.github.com> Co-authored-by: svia3 <svia@amazon.com> Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com> Co-authored-by: Nilesh PS <nps17thatsme@gmail.com> Co-authored-by: Nilesh PS <psnilesh@amazon.com> Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com> Co-authored-by: Jay Goyani <135654128+jgoyani1@users.noreply.github.com> Co-authored-by: Justin <justinm088@hotmail.com> Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com> Co-authored-by: Jonathan Makunga <makung@amazon.com>

…ngestion. (aws#4413) * change: update image_uri_configs 12-13-2023 12:23:06 PST * change: update image_uri_configs 12-13-2023 14:04:54 PST * prepare release v2.200.1 * update development version to v2.200.2.dev0 * fix: Move func and args serialization of function step to step level (aws#4312) * fix: Add write permission to job output dirs for remote and step decorator running on non-root job user (aws#4325) * feat: Added update for model package (aws#4309) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * documentation: fix ModelBuilder sample notebook links (aws#4319) * feat: Use specific images for SMP v2 jobs (aws#4333) * Add check for smp lib * update syntax * Remove unused images * Update repo name and regions * Update account number * Update framework name and check for None distribution * Add unit tests for smp v2 uri * Check enabled * Remove logging * Add cuda version in uri * Update cu121 * Update syntax * Fix black check * Fix black --------- Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> * Fix: Updated js mb compression logic - ModelBuilder (aws#4294) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * documentation: SMP v2 doc updates (aws#1423) (aws#4336) * doc update for estimator distribution art * add note to the SMP doc and minor fixes * remove subnodes * rm all v1 content as documenting everything in aws docs * fix build errors * fix white spaces * rm smdistributed from TF estimator distribution * rm white spaces * add notes to TF estimator distribution * fix links * incorporate feedback * update example values * fix version numbers in the notes Co-authored-by: Miyoung <cmiyoung@amazon.com> * prepare release v2.201.0 * update development version to v2.201.1.dev0 * Fix: Add additional model builder telemetry (aws#4334) * move telemetry code to public * add additional test --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * feature: support remote debug for sagemaker training job (aws#4315) * feature: support remote debug for sagemaker training job * change: Replace update_remote_config with 2 helper methods for enable and disable respectively * change: add new argument enable_remote_debug to skip set of test_jumpstart_estimator_kwargs_match_parent_class * chore: add jumpstart support for remote debug --------- Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> * Update tblib constraint (aws#4317) * Fix: Fix job_objective type (aws#4303) * change: update image_uri_configs 12-21-2023 08:32:41 PST * prepare release v2.202.0 * update development version to v2.202.1.dev0 * Using logging instead of prints (aws#4133) * documentation: update issue template. (aws#4337) * change: update model path in local mode (aws#4296) * Update model path in local mode * Add test * change: update image_uri_configs 12-22-2023 06:17:35 PST * prepare release v2.202.1 * update development version to v2.202.2.dev0 * change: create role if needed in `get_execution_role` (aws#4323) * Create role if needed in get_execution_role * Add tests * Change: More pythonic tags (aws#4327) * Change: More pythonic tags * Fix broken tags * More tags formatting and add a test * Fix tests * Raise Exception for debug (aws#4344) Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * Change: Allow extra_args to be passed to uploader (aws#4338) * Change: Allow extra_args to be passed to uploader * Fix tests * Black * Fix test * Change: Drop py2 tag from the wheel as we don't support Python 2 (aws#4343) * Disable failed test in IR (aws#4345) * Disable failed test in IR * Fix format --------- Co-authored-by: Ruilian Gao <ruiliann@amazon.com> * change: update image_uri_configs 12-25-2023 06:17:33 PST * feat: Supporting tbac in load_run (aws#4039) * feature: support local mode in SageMaker Studio (aws#1300) (aws#4347) * feature: support local mode in SageMaker Studio * chore: fix typo * chore: fix formatting * chore: revert changes for docker compose logs * chore: black-format * change: Use predtermined dns-allow-listed-hostname for Studio Local Support * add support for CodeEditor and JupyterLabs --------- Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * prepare release v2.203.0 * update development version to v2.203.1.dev0 * change: update image_uri_configs 12-29-2023 06:17:34 PST * query hf api for model md (aws#4346) Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> * fix: skip failing integs (aws#4348) Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> * change: TGI 1.3.3 (aws#4335) * prepare release v2.203.1 * update development version to v2.203.2.dev0 * feat: parallelize notebook search utils, add new operators (aws#4342) * feat: parallelize notebook search utils * chore: raise exception in notebook utils if thread has error * chore: improve variable name * fix: not passing region to get jumpstart bucket * chore: add sagemaker session to notebook utils * chore: address PR comments * feat: add support for includes, begins with, ends with * fix: pylint * feat: private util for model eula key * fix: unit tests, use verify_model_region_and_return_specs in notebook utils * Revert "feat: private util for model eula key" This reverts commit e2daefc. * chore: add search keywords to header * fix: change ConditionNot incorrect property Expression to Condition (aws#4351) * fix: Huggingface glue failing tests (aws#4367) * fix: Huggingface glue failing tests * fix: Sphinx doc build failure * fix: Huggingface glue failing tests * fix: failing sphinx tests * fix: failing sphinx tests * fix: failing black check * fix: sphinx doc errors * fix: sphinx doc errors * sphinx * black-format * sphinx * sphinx * sphinx --------- Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> * fix: Add PyTorch 2.1.0 SM Training DLC to UNSUPPORTED_DLC_IMAGE_FOR_SM_PARALLELISM list (aws#4356) * add 2.1 unsupported smddp * formatting * feat: Support custom repack model settings (aws#4328) * change: update sphinx version (aws#4377) * change: update sphinx version * Update sphinx * change: Updates for DJL 0.26.0 release (aws#4366) * change: TGI NeuronX (aws#4375) * TGI NeuronX * Update * Update * fix: add warning message for job-prefixed pipeline steps when no job name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com> * change: JumpStart - TLV region launch (aws#4379) * feat: add throughput management support for feature group (aws#4359) * feat: add throughput management support for feature group * documentation: add doc for feature group throughput config --------- Co-authored-by: Nilesh PS <psnilesh@amazon.com> * change: Enable galactus integ tests (aws#4376) * feat: Enable galactus integ tests * fix flake8 * fix doc8 * trying to see if it works with slow tests * small fixes in import error * fix missing import * try to remove some dependencies from requirement to see if pr test can be fixed * fix flake8 * Enable more tests * Add rerun annotation and further remove dependencies * comment out 2 integ tests * Remove local mode test for now * fix flake8 * prepare release v2.204.0 * update development version to v2.204.1.dev0 * fix: Add validation for empty ParameterString value in start local pipeline (aws#4354) * feat: Support selective pipeline execution for function step (aws#4372) * change: update image_uri_configs 01-24-2024 06:17:33 PST * fix: update get_execution_role_arn from metadata file if present (aws#4388) * fix: Support using PipelineDefinitionConfig in local mode (aws#4352) * fix: remove fastapi and uvicorn dependencies (aws#4365) They are not used in the codebase. Closes aws#4361 aws#4295 * prepare release v2.205.0 * update development version to v2.205.1.dev0 * change: TGI NeuronX 0.0.17 (aws#4390) * fix: Support PipelineVariable for ModelQualityCheckConfig attributes (aws#4353) * feat: Logic to detect hardware GPU count and aggregate GPU memory size in MiB (aws#4389) * Add logic to detect hardware GPU count and aggregate GPU memory size in MiB * Fix all formatting * Addressed PR review comments * Addressed PR Review messages * Addressed PR Review Messages * Addressed PR Review comments * Addressed PR Review Comments * Add integration tests * Add config * Fix integration tests * Include Instance Types GPU infor Config files * Addressed PR review comments * Fix unit tests * Fix unit test: 'Mock' object is not subscriptable --------- Co-authored-by: Jonathan Makunga <makung@amazon.com> * fix: fixed create monitoring schedule failing after validation error (aws#4385) Co-authored-by: Keshav Chandak <chakesh@amazon.com> * Add collection type support for Feaure Group Ingestion. Add TargetStores support for PutRecord and Ingestion. * Remove merge conflicts. * Update the feature definition type * Black formatting * Fix Flake8 formatting * Fix Pylint * Fix Formatting. --------- Co-authored-by: sagemaker-bot <sagemaker-bot@amazon.com> Co-authored-by: ci <ci> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Keshav Chandak <keshav.chandak1995@gmail.com> Co-authored-by: Keshav Chandak <chakesh@amazon.com> Co-authored-by: stacicho <stacicho@amazon.com> Co-authored-by: Teng-xu <67929972+Teng-xu@users.noreply.github.com> Co-authored-by: huilgolr <yoda@ip-10-0-12-252.us-west-2.compute.internal> Co-authored-by: Gary Wang <38331932+gwang111@users.noreply.github.com> Co-authored-by: EC2 Default User <ec2-user@ip-172-16-54-104.us-west-2.compute.internal> Co-authored-by: akrishna1995 <38850354+akrishna1995@users.noreply.github.com> Co-authored-by: Miyoung <cmiyoung@amazon.com> Co-authored-by: Xinyu Xie <xiexinyucrab@126.com> Co-authored-by: Xinyu Xie <xixinyu@amazon.com> Co-authored-by: Evan Kravitz <evakravi@amazon.com> Co-authored-by: martinRenou <martin.renou@gmail.com> Co-authored-by: Duc Trung Le <leductrungxf@gmail.com> Co-authored-by: ruiliann666 <141953824+ruiliann666@users.noreply.github.com> Co-authored-by: Ruilian Gao <ruiliann@amazon.com> Co-authored-by: ananth102 <ananthbashyam1@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <141277478+benieric@users.noreply.github.com> Co-authored-by: Mufaddal Rohawala <mufi@amazon.com> Co-authored-by: amzn-choeric <105388439+amzn-choeric@users.noreply.github.com> Co-authored-by: evakravi <69981223+evakravi@users.noreply.github.com> Co-authored-by: Erick Benitez-Ramos <benieric@amazon.com> Co-authored-by: Sirut Buasai <73297481+sirutBuasai@users.noreply.github.com> Co-authored-by: Sindhu Somasundaram <56774226+sindhuvahinis@users.noreply.github.com> Co-authored-by: Stephen Via <51342648+svia3@users.noreply.github.com> Co-authored-by: svia3 <svia@amazon.com> Co-authored-by: Haixin Wang <98612668+haixiw@users.noreply.github.com> Co-authored-by: Nilesh PS <nps17thatsme@gmail.com> Co-authored-by: Nilesh PS <psnilesh@amazon.com> Co-authored-by: jiapinw <95885824+jiapinw@users.noreply.github.com> Co-authored-by: Jay Goyani <135654128+jgoyani1@users.noreply.github.com> Co-authored-by: Justin <justinm088@hotmail.com> Co-authored-by: Jonathan Makunga <54963715+makungaj1@users.noreply.github.com> Co-authored-by: Jonathan Makunga <makung@amazon.com>

svia3 marked this pull request as ready for review January 11, 2024 22:16

svia3 requested a review from a team as a code owner January 11, 2024 22:16

svia3 requested review from akrishna1995 and removed request for a team January 11, 2024 22:16

svia3 changed the title ~~fix: add warning message for job-prefixed pipeline steps when no job …~~ fix: add warning message for job-prefixed pipeline steps when no job name is provided Jan 11, 2024

rohangujarathi reviewed Jan 11, 2024

View reviewed changes

svia3 force-pushed the job-prefix-warning branch 2 times, most recently from 4c06ae1 to 9167a6a Compare January 11, 2024 22:50

qidewenwhen reviewed Jan 11, 2024

View reviewed changes

svia3 force-pushed the job-prefix-warning branch from 9167a6a to 2ffab7a Compare January 12, 2024 00:23

svia3 force-pushed the job-prefix-warning branch from 2ffab7a to e3e7837 Compare January 12, 2024 00:30

svia3 force-pushed the job-prefix-warning branch from e3e7837 to c3a0d04 Compare January 12, 2024 20:45

svia3 force-pushed the job-prefix-warning branch from c3a0d04 to 7c854b9 Compare January 12, 2024 21:36

svia3 force-pushed the job-prefix-warning branch from 7c854b9 to e783a46 Compare January 12, 2024 23:07

svia3 force-pushed the job-prefix-warning branch from e783a46 to 5d1daa5 Compare January 16, 2024 20:03

svia3 force-pushed the job-prefix-warning branch from 5d1daa5 to d855a8e Compare January 17, 2024 01:02

fix: add warning message for job-prefixed pipeline steps when no job …

e63702e

…name is provided

knikure force-pushed the job-prefix-warning branch from d855a8e to e63702e Compare January 19, 2024 01:21

rohangujarathi approved these changes Jan 19, 2024

View reviewed changes

knikure approved these changes Jan 19, 2024

View reviewed changes

knikure merged commit 93fc2d1 into aws:master Jan 19, 2024
6 checks passed

samruds pushed a commit to samruds/sagemaker-python-sdk that referenced this pull request Jan 30, 2024

fix: add warning message for job-prefixed pipeline steps when no job …

c172675

…name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com>

suryans-commit pushed a commit to suryans-commit/sagemaker-python-sdk that referenced this pull request Feb 5, 2024

fix: add warning message for job-prefixed pipeline steps when no job …

097db73

…name is provided (aws#4371) Co-authored-by: svia3 <svia@amazon.com>

fix: add warning message for job-prefixed pipeline steps when no job name is provided #4371

fix: add warning message for job-prefixed pipeline steps when no job name is provided #4371

Conversation

svia3 commented Jan 11, 2024 • edited Loading

Merge Checklist

General

Tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mufaddal-rohawala commented Jan 11, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 11, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 11, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 12, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 13, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 13, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 16, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 16, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 16, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 16, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 16, 2024

AWS CodeBuild CI Report

codecov-commenter commented Jan 16, 2024 • edited Loading

Codecov Report

mufaddal-rohawala commented Jan 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 17, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 17, 2024

AWS CodeBuild CI Report

svia3 commented Jan 17, 2024

mufaddal-rohawala commented Jan 19, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 19, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 19, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 19, 2024

AWS CodeBuild CI Report

mufaddal-rohawala commented Jan 19, 2024

AWS CodeBuild CI Report

svia3 commented Jan 11, 2024 •

edited

Loading

codecov-commenter commented Jan 16, 2024 •

edited

Loading